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ABSTRACT 



The sensitivity of the ATLAS experiment to the top-quark rare decay via flavour-changing 
neutral currents t — ► Hq (q represents c and u quarks) has been studied at 1 /s=14 TeV in the 
decay mode of it — > HqWb — > WW*q, Wb — > Mz/j, / ± ^6, (l=e, /x). 

The Standard Model backgrounds tt, ttif , WZ and Wii" have been analysed. The signal 
and backgrounds were generated via PYTHIA 5.7 and simulated and analysed using ATLFAST 
2.51. A branching ratio for t — > Hq — > WW*q as low as 2.0xl0 -3 for m# = 150 GeV and 
2.1xl0 -3 for m# = 160 GeV could be discovered at the 5<r level with an integrated luminosity 
of 100 fb" 1 . 
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1 Introduction 



We study the sensitivity of the ATLAS experiment to the branching ratio of the top-quark 
rare decay via flavour changing neutral currents (FCNC) i — > Hq (q = u,c), where H is the 
Standard Model Higgs (m t — 175 GeV, Mz < Mu < 2Myy). The SM prediction for the branching 
ratio is of order Br(i -»• H SM q) « 0.9 • 10" 13 (4 x 10~ 15 ) for 777^=100(160) GeV [1]. Thus, an 
observation of this decay mode at the LHC, would provide a clear signal of new physics beyond 
the SM, such as new dynamical interactions of top quark, multi-Higgs doublets, exotic fermions 
or other possibilities [2,3,4,5]. The recent theoretical estimations of branching is Br(£ — > Hq) ~ 
10~ 5 4- 10~ 4 [5]. We consider the following process it — > HqWb — > WW*q,Wb — > lvli>j,l ± i>b, 
(l=e, fx) for Higgs masses 777/7=150 GeV and m/f =160 GeV in the leptonic decay topology of W 
bosons since in this mass range H — > WW* is the dominant decay mode. 

2 Monte Carlo Event Generation 

It is known, that the dominant mechanism of top quark production at the LHC is it pair 
production via gg, qq — > it. 

For the implementation of the t — > Hq process into PYTHIA 5.7 all individual decay channels 
of the top quark were switched off, except two decay modes t — > Wb and t — > H^s. The last one 
was replaced by the decay of the t — > by replacing of by H and s by c(u) . 

The tt events have been generated by PYTHIA 5.7 at = 14 TeV, m top = 175 GeV 
for two masses of Higgs m H = 150 GeV and m H = 160 GeV with proton structure function 
CTEQ2L. Initial and final state QED and QCD (ISR, FSR) radiations, multiple interactions, 
fragmentations and decays of unstabled particles were enabled. The total cross-section for if 
production was assumed to be cr t f=833 pb (NLO prediction) [6]. 

The following SM backgrounds have been considered for above mentioned process: 

• it -> WbWb -> l+vbl-vb 

• tiH -> W + bW-b, WW* -> l+ubj-ubj+ul-^^ubjjbj+ul-u-j+ubj-i/bjjlu 

• WZ -> + X 

• WH -> l±vWW* -> l ± v,l+vl'v + X 

In case of ii background the third lepton originates from a semileptonic 6 -decay. 
All backgrounds were generated by PYTHIA 5.7. 

The performance of the ATLAS detector was simulated using the fast simulation package 
ATLFAST 2.51 [7], which uses parametrizations of the detector resolution functions. High 
luminosity (KEYLUM=2) option has been invoked. 

The b tagging perfomance was simulated assuming the nominal efficiencies of e& = 50%, 
e c = 10%, €j = 1%. The lepton identification efficiency is e l =0.9. The branching ratios of H — > 
WW* decays have been estimated by the Fortran code HDECAY [8]. Br(H -> WW*)=0.6852 
(0.9160) for 777^=150 GeV (160 GeV). 
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3 Event Analysis 



The experimental signature of ti — > HqWb — > WW*j, Wb — > luluj, Ivb includes three isolated 
leptons, missing transverse momentum due to neutrinos, one b-jet and an additional light jet. 

The leptonic decay mode of W bosons have been considered for the final topology of back- 
grounds. 

Numbers of generated and expected events of signal and backgrounds for two masses of Higgs 
m H = 150 GeV and ran = 160 GeV are presented in Tables l-=-2. In order to discriminate the 
signal from the background processes, the following preselection cuts have been applied: 

• At least three isolated charged leptons (electrons with P T > 5 GeV, \r)\ < 2.5 and muons 
with Pt > 6 GeV, \rj\ < 2.4) are required. Lepton isolation criteria in terms of the distance from 
other clusters AR >0.4 and of maximum transverse energy deposition in cells E T < 10 GeV in 
a cone AR=0.2 around the leptons were applied. 

• Events without at least two jets with P T et > 15 GeV and \rj\ < 5, are rejected. 

The preselection cuts reduce backgrounds approximately from 2% up to 14%, meanwhile 
~ 50% of signal events are survived. 

Then a kinematical cut was applied, requiring the presence of three isolated charged leptons 
with high P T > 30 GeV. This cut certainly includes the presence of one opposite charged lepton 
in event. Such requirement affects (reduces) significantly ti dangerous background, while keeping 
still a significant fraction of the signal. 

The next requirement of the missing transverse momentum in event with p™ ss > 45 GeV 
is a powerful cut for reduction of WZ background (see Fig. 1), though other backgrounds are 
less sensitive to this cut. One can see from Fig. 1, that tiH background is extended to larger 
values of P™ ss than other backgrounds due to four neutrinos in tiH . 

We demand the presence of at least two jets with high P T et > 30 GeV in the pseudorapidity 
region \rf et \ < 2.5, which satisfy the following isolation conditions: ARjj > 0.4 (jet-jet isolation) 
and ARij > 0.4 (lepton-jet isolation). Among the isolated high Pt jets is required the presence 
at least of one tagged b-jet. This cut significantly suppresses WZ background and vanishes 
WH. 

For the dilepton (coming from H decay) mass mu reconstruction it have been required op- 
posite charged lepton pairs with A0 < 1.0 ( the opening angle between the two leptons in 
the transverse plane, measured in rad.), \Arj\ < 1.5 ( the absolute values of the pseudorapid- 
ity difference between the two leptons) and an invariant dilepton mass smaller than 80 GeV 
[9]. The small angular separation between leptons in signal events results from the opposite 
spin orientation of the W pair originating from the decay of the scalar Higgs boson [10]. The 
discrimination between the signal and the most important backgrounds (ti, tiH) is shown for 
the pseudorapidity difference \Ar}\ and the azimuthal difference A0 between the two leptons, 
respectively, in Figures 2 and 3. One can see, that these cuts sufficiently reduce ti background 
and WZ backhground is vanished (see Tables 1-^2). 
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In Fig. 4 are presented the dilepton invariant mass mu distributions for signal and UH, tt 
background events for a Higgs mass of 160 GeV. The solid histogram is for best combinations 
of dilepton pairs for signal and dashed histogram is for Higgs decay product leptons at parton 
level. The best combinations of 11 invariant mass is defined as the closest values to the mean 
of the // dilepton mass distribution of signal at parton level. One can see a good agreement 
between parton level and best combinations of dilepton pairs. The tt and ttH events are reduced 
by « 50%. 

Then the kinematical cuts have been applied on the two invariant masses of: 

a) muj < 110 GeV with light jets P T > 30 GeV and 

b) m lh < 140 GeV with P h T jet > 40 GeV. 

In Fig. 5 are presented the invariant 11 j mass distributions for a Higgs mass of 160 GeV 
for the best combinations of signal (solid histogram), for ttH background (dotted histogram), 
tt background (dashed-dotted histogram) and invariant mass of llq for signal at parton level 
(dashed histogram). One can see an agreement between parton level and the best combinations 
of 11 j. 

The sequence of muj < 110 GeV and rriib < 140 GeV cuts significantly suppresses the ttH 
and tt backgrounds and other remained backgrounds are vanished. 

In Fig. 6 are presented the invariant lb mass distributions for a Higgs mass of 160 GeV 
for the best combinations of signal (solid histogram), for ttH background (dotted histogram), 
tt background (dashed-dotted histogram) and invariant mass of lb for signal at parton level 
(dashed histogram). One can see an agreement between parton level and the best combinations 
of lb for the signal and remained backgrounds. 

The acceptances and the expected rates for signal and background events after applying all 
above mentioned kinematical cuts are summarised in Tables l-i-2 for m#=150 and 160 GeV. 
The sensitivities to Br(t — > Hq) have been calculated similarly as in [11]. Table 3 summarizes 
the branching ratios for the t — > Hq — > WW*q decay at the different levels of applied cuts. One 
can see from Table 3, that Br(£ — > Hq) as low as 2.0xl0~ 3 could be discovered at the 5a level 
for m H =15Q GeV and as low as 2.1xl0~ 3 for m H =160 GeV with an integrated luminosity of 
100 fb" 1 . 

4 Conclusions 

We have studied the ATLAS sensitivity to the FCNC top quark rare decay t — > Hq (q = u, c) 
with H — > WW* at y/s — 14 TeV for an integrated luminosity of 100 fb~ l . The results 
demonstrate that, a branching ratio as low as 2.0xl0~ 3 could be discovered at the 5a level for 
m#=150 GeV and 2.1xl0~ 3 for m H =160 GeV, respectively. 
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Description 
of 
Cuts 


* -> //g 
Signal 


Background Processes 


ttH 


WH 


tt 


WZ 


Nevt 


Eff (%) 


Nevt 


Eff (%) 


Nevt 


Eff (%) 


Nevt 


Eff (%) 


Nevt 


Eff (%) 


Nevt gen. 


10.4K 




110K 




300K 




4M 




200K 




Expect, events 






12950 




430 




3.9M 




39040 




Preselection 


5012 


48.00 


395 


3.05 


56 


13.07 


93611 


2.41 


3552 


9.10 


P™ is > 45 GeV 
Pf > 30 GeV 
P 3 T et > 30 GeV 

Nb-jet > 


249 


2.39 


48 


0.38 





5.13 x 10~ 2 


152 


3.91 x 10~ 3 


5 


1.28 x 10^ 2 


m u < 80 GeV 
Ar] < 1.5 
A<j> < 1.0 


186 


1.79 


22 


0.17 





4.03 x 10~ 2 


38 


9.78 x 10" 4 





1.00 x 10~ 3 


* -> Hq 

muj < 110 GeV" 
P^ > 30 GeV 


87 


0.84 


6 


5.09 x 10" 2 





7.30 x 10~ 3 


7 


1.80 x 10" 4 





5.00 x 10" 4 


t -> 

m ib < 140 CeF 
P£ > 40 GeF 


78 


0.75 


6 


4.64 x 10- 2 





6.70 x 10- 3 


3 


7.72 x 10~ 5 





5.00 x 10" 4 



Table 1: The numbers of events and efficiencies (%) of kinematic cuts applied in sequence for the signal and backgrounds 
for m H = 150 GeV (are not multiplied on the lepton identification efficiency (e') 3 ).The numbers of events after cuts for 
backgrounds are presented according to the expected events. 



Description 
of 
Cuts 


t -> #g 
Signal 


Background Processes 


UH 


WE 


tt 


WZ 


Nevt 


Eff (%) 


Nevt 


Eff (%) 


Nevt 


Eff (%) 


Nevt 


Eff (%) 


Nevt 


Eff (%) 


Nevt gen. 


8.8K 




32.8K 




15K 




4M 




200K 




Expect, events 






15020 




480 




3.9M 




39040 




Preselection 


3943 


45.00 


475 


3.16 


66 


13.70 


93611 


2.41 


3552 


9.10 


P™ s > 45 GeV 
P l T > 30 GeV 
P 3 T et > 30 GeV 

N b - jet > 


218 


2.48 


76 


0.51 





5.33 x 10~ 2 


152 


3.91 x 10~ 3 


5 


1.28 x 10~ 2 


m u < 80 GeV 
Ar] < 1.5 
A(p < 1.0 


170 


1.93 


35 


0.24 





4.67 x 10~ 2 


38 


9.78 x 10" 4 





1.00 x 10~ 3 


t -> #g 

m Hj < 110 Gel/ 
P^ > 30 GeF 


60 


0.68 


12 


8.54 x 10~ 2 





6.70 x 10~ 3 


7 


1.80 x 10" 4 





5.00 x 10" 4 


m ife < 140 Gel/ 
P| > 40 GeF 


59 


0.67 


11 


7.62 x 10~ 2 





5.10 x 10~ 3 


3 


7.72 x 10~ 5 





5.00 x 10~ 4 



Table 2: The numbers of events and efficiencies (%) of kinematic cuts applied in sequence for the signal and backgrounds 
for rriH = 160 GeV (are not multiplied on the lepton identification efficiency (e') 3 ).The numbers of events after cuts for 
backgrounds are presented according to the expected events. 



Cuts 


Sensitivity to Br(t — > Hq) 




m ^ = 150 GeV 


m H = 160 GeV 


t -> Hq 


2.19xl0- 3 


2.43xl0- 3 


m U j < HO GeV, P£ > 30 GeV 






t -> 


2.03xl0~ 3 


2.12X10" 3 


m* 6 < 140 GeF, P£ > 40 GeV 







Table 3: Summary of the results at L — 100 fb 
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Figure 1: The P™ %ss distributions for signal events with m H = 160 GeV and ttH, tt and 
WZ backgrounds. All distributions are normalised to unity. 
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Figure 2: Difference in azimuth between the two leptons for signal events with 
GeV and ti and ttH backgrounds. All distributions are normalised to unity. 
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Figure 3: The absolute values of the pseudorapidity difference between the two leptons 
for signal events with m# = 160 GeV and tt and tiH backgrounds. All distributions are 
normalised to unity. 
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Figure 4: Invariant 11 mass distributions for the signal and background events: the solid 
histogram for the best combinations of signal, the dashed histogram for signal at parton 
level, the dotted histogram for tiH background, the dashed-dotted histogram for ti. The 
arrow indicates the value of cut on dilepton mass. 
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Figure 5: Invariant 11 j mass distributions for the signal and background events: the solid 
histogram for the best combinations of signal, the dashed histogram for signal at parton 
level, the dotted histogram for tiH background, the dashed-dotted histogram for ti. 
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Figure 6: Invariant lb mass distributions for the signal and background events: the solid 
histogram for the best combinations of signal, the dashed histogram for signal at parton 
level, the dotted histogram for tiH background, the dashed-dotted histogram for ti. 
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